Russo & Hopkins 


Measuring Mental Computational Fluency With Addition: A Novel 


Approach 
James Russo Sarah Hopkins 
Monash University Monash University 
<james.russo @ monash.edu> <s.hopkins @ monash.edu> 


Measuring computational fluency, an aspect of procedural fluency, is complex. Many 
attempts to measure this construct have emphasised accuracy and efficiency at the expense 
of flexibility and appropriate strategy choice. Efforts to account for these latter constructs 
through assessing children’s computational reasoning using structured interviews (e.g., 
MAD), are necessarily time-intensive. In this paper, we introduce a novel measure of Mental 
Computational Fluency with Addition (MCF-A) that attempts to incorporate these aspects by 
requiring children to reason from the perspective of another child. We describe results of a 
pilot study using the MCF-A with 169 Year 3 and 4 students. 


Computational fluency has been described as receiving more attention and attracting 
more controversy than perhaps any other topic within mathematics education (Boerst & 
Schielack, 2003). Whether computational fluency should be defined broadly, and 
incorporate conceptual understanding, or narrowly, and focus exclusively on quick and 
accurate recall of basic facts and procedures, has been a particular point of contention 
amongst educational researchers, with the latter definition frequently prevailing (Clarke, 
Nelson & Shanley, 2016). At least in part in response to this debate, the National Council of 
Teachers of Mathematics (NCTM) has consistently sought to explain and clarify what is 
meant by the term. In its Principles and Standards for School Mathematics released nearly 
two decades ago, NCTM describes computational fluency as a “connection between 
conceptual understanding and computational proficiency” (NCTM, 2000, p. 35). The 
Standards go on to emphasise the interplay between concepts and procedures for achieving 
computational fluency: “computational methods that are over-practiced without 
understanding are often forgotten or remembered incorrectly... on the other hand, 
understanding without fluency can inhibit the problem-solving process” (p. 35). Despite 
substantial empirical support for its stance (e.g., the mutual interdependency of conceptual 
and procedural knowledge; Rittle-Johnson, 2017), the NCTM felt the need to further clarify 
what was meant by ‘fluency’ in a 2014 position paper, in response to continued inconsistency 
in the terms interpretation and operationalisation. It introduces the term “procedural fluency” 
to emphasise that fluency is relevant to all areas of mathematics, not just mental computation. 
Procedural fluency is defined as “the ability to apply procedures accurately, efficiently, and 
flexibly... and to recognize when one strategy or procedure is more appropriate to apply 
than another” (NCTM, 2014, p. 1). This definition mirrors that of the National Research 
Council (2001), that emphasises the importance of students using procedures “flexibility, 
accurately, efficiently and appropriately” (p. 116). 

This paper briefly evaluates attempts within educational research to measure fluency 
against these four aspects highlighted by the NCTM (.e., accuracy, efficiency, flexibility, 
and strategy appropriateness), before putting forward a novel measure of computational 
fluency that we assert embodies all of these components. Data from a pilot study trialling 
this new measure, which we term Mental Computational Fluency with Addition (MCF-A), 
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is then presented and discussed. The paper concludes by discussing the practical implications 
of the measures potential use in classrooms, as well as highlighting possible directions for 
future research. 


Attempts to Measure Computational Fluency 


As noted earlier, computational fluency has often been defined narrowly, leading to 
corresponding measures to focus exclusively on speed and accuracy of number fact recall. 

Calhoon, Emerson, Flores and Houchins (2007) refer to the NCTM Standards when 
defining computational fluency, however rather than describe the link between conceptual 
understanding and computational proficiency, they focus on the statement that describes 
computational fluency as “having efficient and accurate methods for computing” (NCTM, 
2000, p. 152). They subsequently interpret this to mean that “to be computationally fluent, 
students must correctly answer mathematics problems at an identified level of difficulty 
within a given time period” (Calhoon et al., 2007, p. 292). Consistent with this narrower 
understanding of fluency, in their study of 224 high school students with mathematics 
disabilities, Calhoon et al. employ the Mathematics Operations Test—-Revised (MOT-R) as 
their fluency measure. This test includes 50 items covering the four operations in whole 
numbers and decimals/ fractions, covering curriculum expectations from Year 2 to Year 6. 
The items are organised according to topic and difficulty, and students are given 10 minutes 
to complete as many items are possible. In their study, Calhoon et al. score the MOT-R such 
that performance is measured by the number of questions correctly answered in the time 
allocated. 

Foster (2017) actually referred to the NCTM position paper when describing 
computational fluency as the ability to perform a procedure “accurately, efficiently and 
flexibly” (NCTM, 2014, p. 1). However, in his two studies of 377 high school students which 
focussed on the number and algebra domain, fluency was again measured narrowly by 
assessing performance on four routine tasks, to be completed within an approximate time 
limit of 10 minutes. Students were given one point for a correct answer, and their overall 
performance was marked out of four. 

Codding, Burns and Lukito (2011) undertook a meta-analysis examining the impact of 
interventions on basic fact fluency. Their analysis included 17 single case-study designs 
covering 55 participants, all of whom were elementary school students identified as 
“struggling with computation fluency” (p. 38). Although the authors do not attempt to define 
explicitly the term computational fluency in their study, part of their inclusion criteria for 
their meta-analysis was that studies include the metric of digits correct per minute in their 
results, and this became the sole outcome variable the authors’ evaluated. Consequently, it 
would appear that Codding et al. (2011), and perhaps the studies that comprised their meta- 
analysis, have again emphasised the accuracy and efficiency aspects of fluency at the 
expense of flexibility and strategy choice. 

Beyond neglecting flexibility and appropriate strategy choice, there are additional issues 
with the above measures of computational fluency worthy of comment. In particular, 
whether the notion of efficiency can be simply equated with speed of recall, or performance 
on a timed-assessment, is problematic. For instance, Star (2005) makes the point that what 
is meant by the term the “most efficient strategy” is quite nuanced and context dependent. 
He asks: “Is the most efficient strategy the one that is the quickest or easiest to do, the one 
with the fewest steps, the one that avoids the use of fractions, or the one that the solver likes 
the best?” (p. 409). Star goes on to propose that there are “subtle interactions among the 


662 


problem's characteristics, one's knowledge of procedures, and one's problem-solving goals” 
that lead an individual to implement a particular procedure in a particular context (p. 409). 

In more general terms, whether time-limited tests have actually been devised to capture 
‘efficiency’ and serve as a proxy measure for the fluency construct, or whether they have 
simply been loosely interpreted as measuring fluency post hoc, is also contentious. Clarke 
et al. (2016) argue that an assessment that is timed is frequently interpreted as a fluency 
measure, when it is actually intended as a broader measure of mathematical performance. 
The authors highlight that often time-limits are imposed on performance assessments simply 
as an artefact of ensuring the measure possesses particular design characteristics to enable it 
“to be administered frequently and repeatedly over time” (p. 79). Arguably, this remains 
problematic because measures which rely on timed tests may also provoke maths anxiety, 
meaning that we are underestimating the true ability of maths anxious individuals when we 
rely on these measures (Ashcraft & Moore, 2009). We suggest that this provides a powerful 
reason to move away from using time-based assessments if it can be demonstrated that there 
are other reliable means of probing the phenomenon of interest. 

It should be acknowledged that we ourselves have recently completed a study that 
claimed to measure fluency by focussing on the completion of routine tasks within a given 
time limit, with one point given for a correct answer, and overall performance marked out 
of 25 (Russo & Hopkins, 2018). Consequently, our above critique should certainly not be 
interpreted as an attempt to undermine the careful work undertaken by our colleagues; rather, 
we acknowledge from our own firsthand experience that measuring computational fluency 
in a manner that comprehensively captures the construct as defined by the NCTM (2014) 
position paper is a complex and difficult business. 

Narrow interpretations and measurements of computational fluency, such as those 
described above, including our own work, are perhaps what have led to concerns than an 
emphasis on fluency may constitute a “threat to reform approaches to the learning of 
mathematics” (Foster, 2017, p. 122). However, there have been attempts to assess 
computational fluency in a manner that values flexibility and strategy choice, in addition to 
accuracy and efficiency. For example, in the early 2000s, a team of Australian researchers 
developed the Early Numeracy Interview, which comprehensively assessed the 
mathematical knowledge expected of students up to around Year 4 (according to most 
curriculum standards) against pre-identified growth points (Clarke et al., 2002). More 
recently, the interview was updated and renamed the Mathematics Assessment Interview 
(MAI; Gervasoni & Perry, 2015). Four of the nine sections of the interview relate to number 
(Counting, Place Value, Addition and Subtraction and Multiplication), and the interview’s 
emphasis on mathematical reasoning, appropriate strategy use, and mental calculation in the 
context of solving arithmetic problems means that these number strands can be perhaps 
broadly equated with assessing computational fluency. 

Although the MAI and its predecessor are valuable tools for classroom teachers to assess 
students’ knowledge and skills, and focus teaching accordingly, there are at least three 
concerns with co-opting the tool as a measure of computational fluency for research 
purposes. Firstly, the interview is undertaken individually with children, and takes between 
30 and 45 minutes depending on a child’s particular responses (Gervasoni & Perry, 2015). 
It is therefore time and resource intensive to administer for research purposes. Secondly, 
these interviews were developed first and foremost as a tool for assessing students in the 
early years (Gervasoni & Sullivan, 2007). Consequently, there are comparatively limited 
items covering multi-digit reasoning. Finally, the purpose of the interview as primarily a 
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formative assessment tool means that many students only complete a small subset of the 
items, limiting its utility as a measure in a research context. 

It should be noted that the large amount of not missing at random data inherent in the 
MAI data collection process does not necessarily render it unsuitable as a measure of 
computational fluency, with sophisticated multiple imputation techniques available to 
researchers to correct for this issue (Rose & Fraser, 2008). However, the a priori requirement 
that such statistically complex adjustments be made certainly impacts the likelihood that it 
will have general appeal as a measure beyond studies which have already collected large 
amounts of MAI data. Specifically, a measure relying on large amounts of imputation would 
lack transparency, require considerable statistical expertise to calculate, be complex to 
interpret, and may face reliability issues in the face of researchers making idiosyncratic 
adjustments to their own individual data sets (e.g., choosing unique sets of imputation 
variables). 

Consequently, there remains a need to develop a reliable, valid and cost-effective 
measure of computational fluency which captures all aspects of the definition of fluency put 
forward by the NCTM, particularly the neglected flexibility and strategy choice components. 
In the following section, we introduce our MCF-A measure and describe how it meets the 
aforementioned need. 


Introducing a Measure of Mental Computational Fluency with Addition 
(MCF-A) 


The MCF-A is a 25-item measure of computational fluency in the context of solving 
addition problems, developed primarily for students in Year 3 and above. At the beginning 
of the MCF-A, children are given the following information: 

You will have about 30 minutes to have a go at the 25 questions on this assessment. As always, we 


just want you to try your best. Before beginning the assessment, we will do two practice questions 
together as a whole class. 


Emma is good at adding numbers. She uses clever strategies to make adding easier. Your job is to try 
and think like Emma did. Explain what Emma did to get the number in the box. 


Children are then presented with the two practice questions, which are completed as a 
whole class (see Figure 1). 


Practice Questions 


For example, to solve 3+4 Emma thinks, that is the same as{4 + 1.What 
numbers did Emma add together to get|6? 


Emma thought... What numbers did Emma add together to get ... 
3+4 


is the same as 6)? 


6+1 


Another example, to solve 8+3 Emma thinks, that is the same as lid + 1. What 
numbers did Emma add together to get [102 


Emma thought... What numbers did Emma add together to get ... 
8+3 

is the same as id? 
liov1 


Figure 1. Practice questions for the MCF-A. 
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Children have to first determine the strategy used by Emma, and represent their thinking 
by recording how Emma obtained the number in the box. Consider the first practice question. 
A child might realise that, when confronted with the problem 3 + 4, there are several potential 
solution strategies. One might directly retrieve the solution (“7”) count-on from the larger 
number (“‘5, 6 ,7””), count-on from the first number (““4, 5, 6, 7”), use the near doubles strategy 
subtracting from the known doubles fact (“4 + 4 — 1”), or use the near doubles strategy 
adding to the known doubles fact (“3 + 3 + 1”). However, it is clear that because Emma has 
realised “3 + 4 is the same as 6 + 1”, Emma has used the near doubles add strategy. The 
child would record “3 + 3” in the space provided, indicating that this is how Emma obtained 
the intermediate solution 6. 

Now consider the second practice question. When confronted with the problem 8 + 3, a 
child may again realise there are a range of possible solution strategies. However, the fact 
that the child knows that Emma has noted that “8 + 3 is the same as 10 + 1” means that he 
or she can be confident that Emma has bridged through 10. Unlike in the first practice 
question, when only one intermediate step was consistent with the information presented to 
students (1.e., 3 + 3), there are in fact two possible means by which Emma could have bridged 
through 10: 8 + 2 or 7 + 3. This question, therefore, has two acceptable answers. 

Across the 25 items included in the MCF-A, a range of mental computation strategies 
are covered to capture the thinking of Emma, including: doubles, near doubles, number 
bonds equalling 10 (and multiples of 10), bridging through 10 (and multiples of 10), 
compensation (overshoot), compensation (change both numbers), jump strategy and split 
strategy. Moreover, the items contain a mix of 1-digit, 2-digit and 3-digit addends, and are 
presented in anticipated order of difficulty (least to most difficult). In terms of calculating a 
total score of computational fluency, children are awarded one point for a correct answer, 
with the measure being scored out of 25. 

We contend that the MCF-A effectively operationalises all four aspects of the 
computational fluency definitions put forward by the NCTM (2001, 2014) and National 
Research Council (2001). Firstly, it requires flexibility, as children need to generate a range 
of potential solution strategies when considering the original addition problem. Secondly, it 
emphasises strategy appropriateness, as children select one strategy from a range of 
alternatives that appropriately mimics Emma’s thinking, and leads them to the same 
intermediate step as Emma. Thirdly, it demands knowledge of efficient strategies, because 
the assessment has been designed such that the intermediate step used by Emma represents 
an efficient means of solving the addition problem. Finally, the child has to accurately recall 
or calculate the appropriate number fact corresponding to the intermediate step arrived at by 
Emma. For example, a child who incorrectly recalls that 3 + 3 = 5 will not be able to identify 
the strategy used by Emma in solving the first practice problem. 


Method 


One hundred and sixty-nine children (boys = 84; girls = 85) in Years 3 and 4 from three 
Melbourne public Primary schools completed the MCF-A as part of a larger study into 
addition strategies. The schools covered a broad range of demographics, with one school 
community relatively advantaged, one relatively disadvantaged, and one school similar to 
the national average. The MCF-A was administered to participants in Terms 2 and Terms 3 
in large groups (5 to 20 students) by the first author. Although students were told they would 
be given “about 30 minutes” to complete the assessment, this was intended and enacted as a 
‘soft’ time limit, with students being given additional time if they required it. From anecdotal 
observations made by the first author, any students requesting more time appeared to make 
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very little progress with the assessment beyond what they had achieved in the first 30 
minutes, suggesting that imposing a ‘hard’ time limit would not have yielded different 


results. 
Results 

Table | 

Descriptive Statistics for MCF-A 
Group N Mean SD Median Range Participation Rate 
Low SES school 31 11.13 6.53 11 0-23 38% 
Medium SES school 60 10.40 6.56 10 0-22 71% 
High SES school 78 13.00 6.37 15 0-22 53% 
Year 3 students 719 11.46 6.50 11 0-23 45% 
Year 4 students 90 11.98 6.60 13 0-23 66% 
Overall 169 «11.73, 6.54 12 0-23 54% 


Table 1 displays the mean scores, median scores, range and standard deviations for the 
MCF-A measure, delineated by different student groups. The overall mean score was 11.7, 
and the median 12, indicating that the middle performing student was correct on about half 
the items. Although the differences between mean scores across year levels might be lower 
than expected, and differences in mean scores across schools at times counter-intuitive (i.e., 
the low SES school outperforming the medium SES school), there was evidence that this 
was a consequence of differing participation rates amongst the groups of children (higher 
achieving students were more likely to consent to participate in the study). 


Table 2 
Individual Items on the MCF-A 


No. 


1? 


2b 


Item 


4+5=8+1 


4+7=10+1 
9+3=13-1 
5+8+5=10+8 
6+3+6=12+3 
8+3+2=10+3 
8+19=28-1 
4+28=30+2 


10+11=20+1 


Correct 
response(s 


) 


6+6 

8+2 

8+20 
2+28; 26+4 


10+10 


% 
Correc 
t 


87.0 


74.0 


No. 


22¢ 
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Item 


48+45=90+3 


36+37=72+1 
44+49=94-1 
76+21=96+1 
56+33=80+9 
22+49=21+50 
23+44=64+3 
67+45+43=110+45 


45+67+82=180+14 


Correct 
response(s 


20+44 
67+43 


40+60+80 


% 
Correc 
t 


39.6 


26.0 


19.5 


43.8 


45.6 


29.0 


35.5 


30.2 


24.9 


10° 28+13=38+3 28+10 56.2 23% =. 235+238=470+3 235+235; 28.4 


b 232+238 

114 21+26=40+7 20+20 59.2 244 456+356=800+12  450+350; 24.9 
b 456+344; 

444+356 
126 11+3+9=20+3 1149 65.1 25 955+445=1000+40 955+45 22.6 
0 
135 6+18+34=40+1 6434; 62.1 
d 8 30+10 


Note. a = doubles or near doubles strategies; b = number bonds equalling (multiples of) 10 or bridging 
strategies; c = jump strategy; d = split strategy; e = compensation strategies. 


Table 2 presents the full list of items from the MCF-A, alongside the corresponding 
strategy, solutions and percentage of students obtaining a correct score on the item. Recall 
that the items were presented in anticipated order of difficulty, such that we would expect 
Item 1 to have the highest percentage of correct scores, and Item 25 the lowest percentage. 
This was generally the case, with the exception of the compensation items, which were more 
difficult than anticipated. For example, consider Item 3, which we expected to be the third 
easiest item, and for approximately three-quarters of students to respond with a correct 
answer. In fact, less than one in five students were able to correctly respond to the item: 
“Emma thought 9+3 is the same as 13-1; how did she get the 13?” by identifying that Emma 
had added 10 to the 3 to make 13, before “paying back’ the 1 to get her final answer (that is, 
she applied the compensation — overshoot strategy). This suggests that the compensation 
items (Items 3, 7, 16 and 19) should have been included later in the assessment. 

Finally, the Cronbach Alpha for the MCF-A measure was excellent (a= 0.92). Moreover, 
Cronbach Alpha could not have been improved through removing any of the 25 items. 


Discussion and Future Directions 


As highlighted in our introduction, previous efforts to measure computational fluency 
for research purposes have either defined the construct narrowly (e.g., Calhoon et al., 2007), 
or been time-intensive to administer (e.g., Gervasoni & Sullivan, 2007). We have attempted 
to demonstrate that it is possible to create a reliable, time-efficient measure of computational 
fluency with addition (MCF-A) that operationalises all four aspects of the definition put 
forward by the NCTM (2001, 2014); that is, flexibility, strategy appropriateness, efficiency 
and accuracy. There are, however, a number of future research directions that warrant 
consideration. 

First, although we have presented evidence that the MCF-A is a valid and reliable 
measure, the next step would be to expose it to a more rigorous evaluation of its properties 
as a measure through Rasch analysis. This will allow the MCF-A to be further refined and 
improved. Second, there is a need to apply these same design principles to develop a measure 
of computational fluency within other number domains, such as multiplication. Third, the 
correlation between scores on the MCF-A and other less comprehensive measures of 
computational fluency (e.g., recall of addition facts) should be examined. We are intending 
to explore these three ideas in the near future. 
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Finally, in terms of its practical implications, although the MCF-A measure described in 
this paper has predominantly been developed with a research purpose in mind, it may be of 
value to classroom teachers as a formative assessment tool. As noted by Russell (2000) 
teachers “cannot simply accept any student method” but rather they “need to analyse what a 
student’s procedure reveals about their underlying understanding so that (they) can plan the 
next steps in instruction” (p. 157). The MCF-A meets this need through offering insight into 
which addition strategies students are able to execute with a high degree of proficiency, and 
which strategies require further exploration and consolidation. For instance, within our 
sample, results indicate that many students need more exposure and practice with 
compensation strategies in particular. Thus, although certainly not intended to replace tools 
designed for more comprehensive formative assessment (e.g., MAT), the MCF-A is arguably 
of more value to teachers than other standardised measures used as proxies for fluency (e.g., 
MOT-R), as the latter tend to solely emphasise outcomes (i.e., correct or incorrect) over 
process (e.g., strategy choice). 


References 


Ashcraft, M. H., & Moore, A. M. (2009). Mathematics anxiety and the affective drop in performance. Journal 
of Psychoeducational Assessment, 27(3), 197-205. 

Boerst, T. A., & Schielack, J. F. (2003). Toward understanding of "computational fluency". Teaching Children 
Mathematics, 9(6), 292. 

Calhoon, M. B., Emerson, R. W., Flores, M., & Houchins, D. E. (2007). Computational fluency performance 
profile of high school students with mathematics disabilities. Remedial and Special Education, 28(5), 292- 
303. 

Clarke, B., Nelson, N., & Shanley, L. (2016). Mathematics Fluency—More than the Weekly Timed Test. In 
K. D. Cummings & Y. M. Petscher (Eds.), The Fluency Construct (pp. 67-89). New York, NY: Springer. 

Clarke, D.M., Cheeseman, J., Gervasoni, A., Gronn, D., Horne, M., McDonough, A., . . . Clarke, B. (2002). 
Early numeracy research project final report. Melbourne: Australian Catholic University. 

Codding, R. S., Burns, M. K., & Lukito, G. (2011). Meta-analysis of mathematic basic-fact fluency 
interventions: A component analysis. Learning Disabilities Research & Practice, 26(1), 36-47. 

Foster, C. (2018). Developing mathematical fluency: Comparing exercises and rich tasks. Educational Studies 
in Mathematics, 97(2), 121-141. 

Gervasoni A., Perry B. (2015) Children’s Mathematical Knowledge Prior to Starting School and Implications 
for Transition. In B. Perry, A. MacDonald & A. Gervasoni (Eds.), Mathematics and Transition to School: 
Early Mathematics Learning and Development. Singapore: Springer. 

Gervasoni, A., & Sullivan, P. (2007). Assessing and teaching children who have difficulty learning arithmetic. 
Educational & Child Psychology, 24(2), 40-53. 

National Research Council Staff (NRC), (2001). Adding it up: Helping children learn mathematics. 
Washington, DC, United States: National Academies Press. 

National Council of Teachers of Mathematics (NCTM) (2000). Principles and Standards for School 
Mathematics. Reston: NCTM Ed. 

National Council of Teachers of Mathematics (NCTM) (2014). Procedural fluency in mathematics: A position 
of the National Council of Teachers of Mathematics. Reston: NCTM Ed. 

Rittle-Johnson, B. (2017). Developing Mathematics Knowledge. Child Development Perspectives, 11(3), 184- 
190. 

Rose, R. A., & Fraser, M. W. (2008). A simplified framework for using multiple imputation in social work 
research. Social Work Research, 32(3), 171-178. 

Russell, S. J. (2000). Developing computational fluency with whole numbers. Teaching Children Mathematics, 
7(3), 154-158. 

Russo, J., & Hopkins, S. (2018). Teaching primary mathematics with challenging tasks: How should lessons 
be structured? The Journal of Educational Research. 

Star, J. R. (2005). Reconceptualizing procedural knowledge. Journal for Research in Mathematics Education, 
36(5), 404-411. 


668 


